The following relates to the time series processing arts, monitoring arts, control arts, and related arts.
In system monitoring applications, events are recorded as a function of time. Events may occur on a predetermined schedule, such as a diagnostic sub-system that produces a diagnostic report (i.e., event) at regular intervals. Additionally or alternatively, events may occur at random or pseudorandom times, such as a manual or automatic report of a detected component malfunction.
When an event is received, it is desirable to rapidly determine or predict an appropriate response (which in some instances may be no response at all). In one way of viewing the problem, the event is classified with the output of the classifier being the predicted response. The predicted appropriate response is implemented (in the case of “no response at all” the “implementation” is to do nothing). Eventually, the correctness or incorrectness of the predicted response (that is, the “true” response) is determined.
To provide a more concrete example, consider a help desk maintained by a printing system manufacturer. The events in this case correspond to manual or automatic reports of printing system malfunctions or, more generally, manual or automatic reports of apparently anomalous printing system behavior. In some embodiments, the predicted response can be modeled as a binary decision: either “investigate” or “do nothing”. The “true” response is later determined.
In the case of a predicted response of “investigate”, if the investigation results in some remedial action then the predicted response of “investigate” was correct (that is, the true response was indeed “investigate”). On the other hand, if the investigation results in no remedial action then the predicted response of “investigate” was not correct (that is, the true response was “do nothing” which would have been the more efficient response).
In the case of a predicted response of “do nothing” if no further indication of a problem is received over a sufficiently long period of time then the predicted response of “do nothing” can be assumed to have been correct. On the other hand, if subsequent events (e.g., subsequent reports of the same or similar anomalous printing system behavior) ultimately result in some remedial action being taken, then the predicted response of “do nothing” was incorrect (that is, the true response was “investigate”).
In some such actual help desk operations, it has been found that the true response was “do nothing” in up to 80% of all cases. On the other hand, it is not advisable to “do nothing” in response to a customer or client reporting a genuine problem. Thus, efficient prediction of the appropriate response can result in a large improvement in efficiency of help desk operation.
It can be advantageous to adjust the predictor or classifier based on the true responses, so as to increase accuracy. Toward this end, an “immediate disclosure” assumption is sometimes made, whereby it is assumed that the true response is known shortly after issuance of the predicted response, and before receipt of the next event. The immediate disclosure assumption is computationally convenient—however, it is not realized in many practical system monitors. In the illustrative printing system help desk application, for example, the immediate disclosure assumption may or may not hold in a particular instance in which the predicted response was “investigate”, depending upon how rapidly the investigation is concluded compared with the time interval to the next event. The immediate disclosure assumption generally does not hold when the predicted response is “do nothing”, because the true response is not known in those instances until some time passes in order to determine whether subsequent events do or do not report the same or a similar problem.
The following discloses methods and apparatuses for system monitoring and other time series processing which accommodate delayed disclosure.